Intelligent agent remote tracking of chemical and biological clouds

ABSTRACT

An intelligent agent-accomplished detection and tracking system responsive to sensed characteristics in a cloud of chemical or biological warfare agent(s) dispersed over a geographic area of the earth. The intelligent agent elements of the invention provide an organized and repeated comparison of signal data extracted from overhead dispersed conventional sensors of the chemical or biological agent material and accomplish communication with other agents and the outside world using a common flexible communication language. The intelligent agent elements are disposed in hierarchical arrays having at least lower level, mid level and upper level locations and are inclusive of multiple path forward and feedback agent communications.

RIGHTS OF THE GOVERNMENT

The invention described herein may be manufactured and used by or forthe Government of the United States for all governmental purposeswithout the payment of any royalty.

BACKGROUND OF THE INVENTION

Currently, there is no known wide-area, permanently deployed capabilityfor detecting or tracking chemical attacks on the United States or othernations. Under these conditions, there can be little notice or warningof such an attack nor ability to track the spread of a cloud of theattack agent(s). A system capable of providing warning and tracking formass attacks by detecting the release, dispersion, and drift of achemical cloud is however believed technically feasible. Such a systemcan provide warning from the time of release until an attack agent clouddisperses to the point of no longer posing a threat. Current eventsdemonstrate the need for a capability for detecting such an attack andtracking it, at a level of capability above merely following theprogress of the attack by monitoring those who have been affected. Whileno one presently available technology can provide the capability forsuch warning and tracking, a combination of remote sensors, some ofwhich can provide response to both chemical and biological agents, canprovide a high degree of confidence, specificity, and sensitivity sothat a space-based system can effectively function as an attackearly-warning system. The present invention is believed to provide asignificant part of such a system.

As may now be or subsequently become apparent herein the word “agent”may appear in two differing contexts with respect to the presentinvention; the first of these contexts, as may be appreciated from theabove recited invention title for example, relates to the buildingblocks appearing in portions of the disclosed chemical detection systemarchitecture. The second context for this word “agent” of course relatesto the chemically reactive material used by an enemy to inflict harm.Since the word “agent” appears to be proper and of current usage in eachof these contexts and appears unlikely to cause reading orinterpretation confusion no effort to substitute a less desirablesynonym is made in this description.

SUMMARY OF THE INVENTION

The present invention provides an aircraft, spacecraft or elevatedlocation-based system for detecting and tracking large scale chemicaland biological warfare attack agent dispersals.

It is therefore an object of the present invention to provide anelevation-based system to detect and track chemical and biologicalwarfare agent attacks.

It is another object of the invention to provide the algorithms andprocessing concepts needed by an overviewing system to detect and trackchemical and biological warfare agent attacks.

It is another object of the invention to provide a communication systemusable between signal processing intelligent agents organized in ahierarchy.

It is another object of the invention to provide for the processing ofdata generated by a plurality of sensors in response to a large-scalerelease of chemical and/or biological warfare agents.

It is another object of the invention to provide a hierarchicalorganized array of intelligent agent decision elements suitable for usein large mapping projects.

These and other objects of the invention will become apparent as thedescription of the representative embodiments proceeds.

These and other objects of the invention are achieved by the method ofdetecting and tracking chemical and biological warfare agent plumes overa geographic area, said method comprising the steps of:

disposing a plurality of diverse input condition responsive warfareagent sensor elements above said geographic area of detecting andtracking;

coupling an output from each of said diverse input condition responsivewarfare agent sensor elements to a sensor-specific first levelintelligent agent decision element in a hierarchical array ofintelligent agent decision elements;

extracting, in said sensor specific first level intelligent agentdecision element, cloud-related signal indicia from said output, saidextracting including applying segmented output data and first levelintelligent agent knowledge base data to a decision engine algorithmportion of said sensor-specific first level intelligent agent decisionelement to generate a sensor-specific first level intelligent agentdecision element eXtensible markup language document type definitionoutput;

said sensor-specific first level intelligent agent decision elementeXtensible markup language document type definition output alsoincluding a cloud identification output and a warfare agent sensorelement output quality-determined first confidence signal;

applying said eXtensible markup language document type definition outputof said sensor-specific first level intelligent agent decision elementto a plurality of second and successive other mid level intelligentagent decision elements in said hierarchical array of intelligent agentdecision elements;

evaluating in said mid level intelligent agent elements cloud, weathercondition, ground condition and cloud duplicate related outputs receivedvia said mid level eXtensible markup language document type definitionoutputs of said first level intelligent agent elements;

said evaluating step including a determination of possible natural causefor said cloud-related outputs, a determination of said cloud-relatedoutputs being a warfare agent plume and a determination of a secondconfidence signal, said determinations being encoded into an intelligentagent decision element eXtensible markup language document typedefinition output;

connecting said second intelligent agent decision element eXtensiblemarkup language document type definition output to a plurality of upperlevel intelligent agent decision elements in said hierarchical array ofintelligent agent decision elements;

determining in said plurality of upper level intelligent agent decisionelements in said hierarchical array of intelligent agent decisionelements an occurrence of a warfare substance attack event and a courseof travel for said warfare agent plume in said geographic area, saiddetermining including determining a third confidence signal level andencoding of said determining step signals into an intelligent agentdecision element eXtensible markup language document type definitionoutput.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings incorporated in and forming a part of thespecification, illustrate several aspects of the present invention andtogether with the description serve to explain the principles of theinvention. In the drawings:

FIG. 1 shows a hierarchical agent configuration usable in embodiments ofthe invention.

FIG. 2 shows a hierarchical agent configuration inclusive of blackboardcommunication usable in embodiments of the invention.

FIG. 3 shows an overall or gross architectural representation of anintelligent agent usable in embodiments of the invention.

FIG. 4 shows a more detailed representation of an intelligent agentusable in embodiments of the invention.

FIG. 5 shows the basic data processing flow for an embodiment of theinvention including the steps and decisions made to locate and track achemical or biological agent cloud.

FIG. 6A shows a first part of operation of an Environment Analyst agent.

FIG. 6B shows a second part of operation of an Environment Analystagent.

FIG. 7 shows operation of a Current World Model agent.

FIG. 8 shows operation of a Expected World Model agent.

FIG. 9 shows operation of a Other Sensors Input Analyst.

FIG. 10A shows a first part of operation of a Feedback Analyst.

FIG. 10B shows a second part of operation of a Feedback Analyst.

DETAILED DESCRIPTION OF THE INVENTION

The following description involves discussions of combination of achemical warfare agent, a biological warfare agent and a plurality ofintelligent agent algorithm architectures. In order to avoid confusionamong these three differing “agents”, especially between either of thefirst two of these agents and the latter agent, the word “substance” isused to the best degree reasonably possible in referring to the firsttwo of these agents, i.e., to agents described as a chemical warfaresubstance or a biological warfare substance or generically as a“substance”. No change in meaning is intended by this semanticclarification however.

The present invention therefore concerns an intelligent agentarchitecture and processes usable for the detection and tracking of alarge-scale chemical substance release by way of sensors placed at highaltitude or in space. The invention includes a description of the sensorsignal processing architecture and its operation and the overallprocesses used to allow a combined sensor suite to detect, warn-of, andtrack a chemical or biological agent cloud. In the invention, chemicalor biological agent attack detection and warning is based upon uniquephysical properties that can be identified remotely and then correlatedto provide a high probability of accurate identification. Knowntechnologies can be used as sensors and can consist of, for example,conventional infrared sensors, radar, synthetic aperture radar, andultraviolet sensors that detect properties of a cloud (both reflectiveand transmissive properties) as well as the temperature, moisture, andvisual characteristics of any cloud. The sensors can be disposed above ageographic area of interest by way of terrain features, aircraft orsatellites or other means as known in the art. The detected propertiesenable early and rapid detection of a cloud. Weather information can beincluded in the analysis of sensor data. Taken together, these sameproperties also allow a cloud to be tracked until it has dissipated.

An intelligent agent is a software system that contains knowledge abouta problem domain, is capable of making decisions relative to thatdomain, possesses artificial intelligence, and knows something about thecontext of any request made of it. An intelligent agent also has theknowledge needed to process the information given to it. Finally, suchagents can operate as a society to enable rapid data interchange andcorrelation of analysis results and have the ability to exchangeinformation within the context of a “contract” that specifies the form,content, and conditions under which information is exchanged. Anintelligent agent functions continuously and autonomously within anenvironment that may be inhabited by other agents. Generally,intelligent agents can learn from experience, and by virtue of theirability to communicate they can cooperate to perform tasks.

Intelligent agents have three defining characteristics, agency,intelligence, and communication. Agency is the capability of an agent toact independently in the pursuit of the accomplishment of a task.Intelligence encompasses the degree of reasoning in the intelligentagent's objectives. Communication allows intelligent agents to shareinformation and analyses, enables a problem to be partitioned betweenagents, and allows an intelligent agent system to scale its performancewith an increase in computational power availability and with increasednetwork bandwidth.

The architectural outline of the invention discloses a set ofintelligent agents that communicate using either a hierarchical paradigmarrangement or combined blackboard and hierarchical paradigmarrangement. Each intelligent agent produces as assessment of someaspect of the environment based upon its inputs. The assessment consistsof, at a minimum, a rating and an XML-based description of the inputsand a textual description of the outputs. In the invention, the conceptis to attach a rich but constrained semantics to the analytical outputsfrom the agent so that other intelligent agents can use the textualdescription and numerical rating to further develop their assessments.The letters XML used herein are an abbreviated reference to theeXtensible Markup Language, a descriptive media found helpful indescribing the invention. Additional information regarding theeXtensible Markup Language is included in the discussion and listing ofreferences appearing in Appendix 1 immediately preceding the claims ofthe present document.

Two types of intelligent agents appear in the present invention. Thefirst type comprises agents that are responsible for directly analyzingsensor outputs. The second type comprises agents that primarily analyzethe outputs from other intelligent agents. Decision-making within anagent is performed by one or more decision engines, as discussed below.Intelligent agents that directly process sensor outputs have as theirprimary tasks segmenting the data in order to extract features,determining the location of each segment, and assigning property valuesto each segment. These agents are found at the bottom of the hierarchyand are used to locate features in the environment that may be ofchemical or biological plumes nature. These features are called segmentsbecause they are identified using a segmentation process. Theintelligent agents located higher in the hierarchy are used to analyzethe outputs from other intelligent agents and have as their task thedetection of patterns within the data, determining correlations betweenpatterns, detecting correlations between segments and properties betweendifferent sensors, and determining if a previously detected correlationbetween segments or properties has changed location. Once a segment hasbeen identified as a chemical or biological cloud, it is called a plume.The intelligent agents highest in the hierarchy have as their tasks thedetermination of whether or not one or more chemical or biologicalclouds have appeared in the environment and its/their motion.

To enable rapid analysis and correlation of sensor signals, either ahierarchical or hybrid hierarchy-blackboard intelligent agent system canbe used as the overall architecture for the invention. When a hierarchyof agents is used, as shown in the system 100 of FIG. 1, the intelligentagents lowest in the hierarchy interface directly with a sensor and areused to analyze a single property as reported by a sensor signal. Theintelligent agent 104 in FIG. 1 is used to analyze the data from theinfrared sensor 102 for example. The results of the analysis by theintelligent agents lowest in the hierarchy are transmitted to otherintelligent agents higher in the hierarchy, as at 106 in FIG. 1. Theseagents correlate the analyses from a number of lower-level agents todetermine if an attack has occurred. The agents in the higher parts ofthe hierarchy, the agents at 108 and 110 for example, can also use thecorrelated results to track a chemical or biological agent cloud once itis detected. The agents in the higher parts of the hierarchy also feedback analytical results from higher levels of the hierarchy to the lowerlevels of the hierarchy. Feedback consists of control inputs for thelower level agents and of analytical results in order to aid the lowlevel agents in their task of locating and tracking a cloud. Feedback isalso used to vary the segmentation settings, threshold settings anddecision criteria used by an agent to determine if a biological orchemical cloud is present. Feedback can go from any intelligent agent ata higher level of the hierarchy to any intelligent agent at a lowerlevel of the hierarchy as is represented at 112 and 114 in FIG. 1, butin practice feedback is primarily routed to intelligent agents at thelevel of the hierarchy that is one level below as at 112 in the FIG. 1system.

The size of a cloud that can be detected and tracked by a system of theFIG. 1 type is constrained only by the sensitivity of the sensors usedat 102 and 103 for example. For the sake of clarity, the FIG. 1 drawingshows only the outputs from one level of the hierarchy being sent to asingle agent at the next level, but in practice each agent at a givenlevel communicates with every agent at the next level of the hierarchyas is represented at 118 and 120 in FIG. 1 for example. The intelligentagents at all but the lowest level of the hierarchy are used tocorrelate, analyze, and consolidate outputs from lower levels in orderto determine if a chemical or biological cloud is present and if so itssize and direction of motion. Within the invention, each intelligentagent can use any decision-making system to analyze inputs provided by asensor or other intelligent agents, or both. Therefore, Bayesiannetworks, fuzzy logic, rules, case based reasoning, probabilisticreasoning, genetic algorithms, or other reasoning techniques may be usedby any intelligent agent in the invention.

Correlation can be performed using a statistical technique, such as aweighted sum or a linear correlation. The intelligent agents in thehighest level of the hierarchy interface to an external communicationsystem as indicated at 116 in FIG. 1 to handle aspects of communicatingthe outputs from the hierarchy.

Blackboard System

In the arrangement of the invention wherein where a blackboard system isused, as shown at 200 in FIG. 2, an intelligent agent is again dedicatedto each sensor and is used to analyze a singlec property in a sensorsignal. The intelligent agent architecture is again overall a hierarchy.However, the first two levels of the hierarchy communicate by postingtheir outputs to a common blackboard 202. The results of this analysisare posted to blackboard 202 for other intelligent agents to use as partof their analysis to determine if a chemical or biological cloud ispresent and to track the cloud. The higher levels of the hierarchy havethe same operational responsibilities as the intelligent agents in theFIG. 1 first configuration of the invention.

FIG. 2 thus shows the invention configured to use a common blackboardfor communication between intelligent agents in the first two levels ofthe hierarchy of intelligent agents. In this configuration of theinvention feedback between intelligent agents at the two lowest levelsof the hierarchy is accomplished using the blackboard, any informationto be passed from one intelligent agent to another is placed on theblackboard where any other agent may note its presence and act upon theinformation. Feed back from higher levels of processing to the two firstlevels of processing is also accomplished by the higher level agentsposting their feed back results on the blackboard as is shown at 204 forexample. Feed back between intelligent agents from the highest levels ofthe hierarchy is accomplished in the same manner as in the firstconfiguration for the invention. That is, feed back at the upper levelsof the hierarchy can go from any intelligent agent at a higher level ofthe hierarchy to any intelligent agent at a lower level of the hierarchyas is represented at 206 for example, but in practice feedback isprimarily routed to intelligent agents at the level of the hierarchythat is one level below as is represented at 208 for example.

For the sake of clarity, FIG. 2 shows only the outputs from one level ofthe hierarchy being sent to a single agent at the next level, but inpractice each agent at a given level communicates with every agent atthe next level of the hierarchy above the level in the hierarchy wherethe blackboard is placed. As in the case of the FIG. 1 hierarchy, thesize of the chemical or biological cloud that can be detected andtracked is constrained only by the sensitivity of the sensors. As in theFIG. 1 configuration of the invention, within this configuration of theinvention, each intelligent agent is free to use any decision-makingsystem to analyze its inputs from either a sensor or from otherintelligent agents, or both. Therefore, Bayesian networks, fuzzy logic,rules, case based reasoning, probabilistic reasoning, geneticalgorithms, or other reasoning techniques may be used by any intelligentagent in the system. Correlation can be performed using a statisticaltechnique, such as a weighted sum or a linear correlation. Theintelligent agents in the highest level of the hierarchy interface to anexternal communication system that handles all aspects of communicationof the outputs from the hierarchy.

Within the invention, each agent outputs an assessment based uponsensors'outputs or on the outputs from other intelligent agents. Theassessment from an intelligent agent contains a rating, an assessment(in text), a confidence level for the assessment, and other informationas described below. The output from an intelligent agent in theinvention is constrained to what is permitted in the eXtensible MarkupLanguage (XML) Document Type Definition (DTD) defined for the inventionand contains a description of the intelligent agent's analysis andassessment so other agents can use both the analysis and assessment asinputs for their own reasoning and assessments. The output from eachintelligent agent in the invention is expressed using the eXtensibleMarkup Language. Intelligent agents can output their assessment, aconfidence in the assessment, and the raw values for any or all of thesensors that it used to make the assessment along with otherinformation.

At the lowest level of the hierarchy are the sensor specific intelligentagents. These agents have as their primary responsibility the task ofextracting information that can be indicative of a chemical orbiological agent attack from the data provided by its sensor. As eachsensor or data source generates data it passes the data to its dedicatedintelligent agent. Once the data arrives at the agent, the agentperforms segmentation on the data to look for signs of a chemical orbiological cloud and attempts to isolate any signs of a cloud based uponthe agent's mission tasking by using the information in its knowledgebase. Each low level agent is tasked with making a determination aboutthe characteristics of the observed world based upon the output of onesensor, with the option of using decision engines to do so.

The output from an intelligent agent can consist of a composite of theanalyses of all of the decision engines and can also include theanalysis and confidence level for each decision engine in theintelligent agent. If a decision is allowed to make an output to thehierarchy, it must also include a confidence value with its assessment.Each intelligent agent or agent society continues its analysis on thedata until it either can make a conclusive determination concerning thepresence of a cloud or new data arrives from the sensor. The low-levelintelligent agents are tasked with identifying clouds and areresponsible for attaching cloud identifiers to clouds. If an assessmentis made, then the intelligent agent makes a determination of theconfidence value for the assessment. The confidence value can bedetermined using a look up table, a formula, or other means that takesinto account the quality of the data. Then, the agent completes the DTDby filling in those portions for which it has data and then sends theDTD to all of the agents in the next level of the hierarchy.

The mid-level agents in the hierarchy have as their inputs data providedby intelligent agents at lower levels in the hierarchy. These inputsfrom the lower levels can be optionally consolidated using differentweighting factors and the raw sensor data as provided by theirintelligent agents. The output from a mid-level intelligent agent canconsist of a composite of the analyses of all of its decision enginesand can also include the analysis and confidence level for each decisionengine in the intelligent agent. If a mid-level agent makes an output tothe hierarchy, it must also include a confidence value with itsassessment. Because the mid-level intelligent agents have a multitude ofinputs from lower-level intelligent agents, they also consider theweather and ground conditions as part of their analysis in order to ruleout natural causes for the observed event.

Mid-level agents correlate and consolidate cloud identificationsprovided to them by lower level agents and are responsible fordetermining when a given cloud has been identified by more than oneagent and then consolidating and correlating this multiple recognitionof a cloud into a single instance. Mid-level intelligent agents areallowed to make a determination if a chemical or biological agent plumehas been detected, if such a determination is made then a confidencevalue for the determination must also be made and transmitted in theDTD. Each mid-level intelligent agent continues its analysis on the datauntil it either can make a conclusive determination concerning thepresence of a cloud or new data arrives from the sensor. If anassessment is made, then the intelligent agent computes a determinationof the confidence value for the assessment. The confidence value can bedetermined using a look up table, a formula, or other means that takesinto account the quality of the data. Then, the agent completes the DTDby filling in those portions for which it has data and then sends theDTD to all of the agents in the next level of the hierarchy—or posts theDTD to the blackboard.

TABLE 1 XML Document Type Definition Used by Intelligent Agents at theHierarchy Top <?XML version=“1.0” encoding=“UTF-8”?> <!DOCTYPE DETECTIONOUTPUT SYSTEM [ <!ELEMENT DETECTION OUTPUT Attack occurring Attackassessment confidence ? Number of plumes ? (Plume identifier plumeconfidence plume location Main axis Minor axis Change in main axisChange in minor axis Direction of motion Velocity of motion Plume meanheight Plume maximum height Change in plume height)* > <!ELEMENT Attackoccurring  (#PCDATA)> <!ELEMENT Attack assessment confidence (#PCDATA)><!ELEMENT Number of plumes  (#PCDATA)> <!ELEMENT Plume identifier (#PCDATA)> <!ELEMENT plume confidence  (#PCDATA)> <!ELEMENT plumelocation  (#PCDATA)> <!ELEMENT Main axis  (#PCDATA)> <!ELEMENT Minoraxis  (#PCDATA)> <!ELEMENT Change in main axis  (#PCDATA)> <!ELEMENTChange in minor axis  (#PCDATA)> <!ELEMENT Direction of motion (#PCDATA)> <!ELEMENT Velocity of motion  (#PCDATA)> <!ELEMENT Plumemean height  (#PCDATA)> <!ELEMENT Plume maximum height  (#PCDATA)><!ELEMENT Change in plume height  (#PCDATA)> </DETECTION OUTPUT>

At the top of the hierarchy lie the intelligent agents that have theresponsibility for determining if an attack has occurred and fortracking a chemical or biological cloud or clouds once they aredetected. The output for these agents is formatted according to the DTDshown in Table 1 above. The output from the highest level agentsconsists of a notification of whether or not a chemical or biologicalcloud plume has been detected, how many plumes are detected, a plumeidentifier, a plume location, a confidence factor for the detection ofeach plume, the major and minor axis for each plume, direction andvelocity of each plume, and plume height. The attack occurring entry canhave either a yes or no value, with a default value of no. The attackassessment confidence entry only occurs when the Attack occurring entryhas a value of yes and gives a confidence value that an attack isoccurring. The number of plumes entry provides the number of plumesdetected and appears only when the Attack occurring entry has a value ofyes. Then, for each plume, there is a plume identifier, a confidencefactor value that the plume is a chemical or biological cloud, theplume's coordinates, the main and minor axis for the plume, the changein main and minor axis for this plume since the last report, thedirection and velocity of motion for the plume, the mean and maximumheight for the plume, and the change in plume height since the lastreport.

As shown in FIG. 3, there are four major components within eachintelligent agent in the invention: the Physical RepresentationComponent, PRC, at 303, the Cognitive Component, CC, at 302, the AgentInterface, AI, at 301, and the Knowledge Base at 304. The PRC at 303contains a model of the external environment, including terrain, sunposition, moon position, and major bodies of water within the sensor'sfield of view. The PRC contains static environment information, such asinformation concerning terrain and the location of major bodies ofwater, semi-static environment information, such as informationconcerning weather and soil moisture content, and dynamic data, such asthe reflectance of a cloud, soil temperature, and the Sun's position.The Knowledge Base at 304 contains two types of information. The firsttype of information is that related to the mission for the intelligentagent, that is, the exact tasking for the agent within the invention.The tasking is a description of the objective of the analysis to beperformed by the agent and the inputs the agent should use to performthe analysis. The tasking determines the type of information the agentneeds as an input and the information it can produce as an output.Information about the tasking is exchanged between the Mission KnowledgeBase and the Agent Interface so that the Agent Interface can extract theinformation that the agent needs and so that feedback concerning theagent's performance can be placed into the Mission Knowledge Base foruse by the Decision Engines in the Cognitive Component (at 302). Theother type of information in the knowledge base is agent specific andholds the knowledge needed by each agent for it to perform its task.Information in this portion of the knowledge base helps the agent toanalyze the information arriving from the PRC and also helps it to makeassessments concerning the confidence in an assessment.

The Cognitive Component 302 of the intelligent agent contains one ormore decision engines. The decision engines use the informationcontained in the Knowledge Base 304 and the information in the PhysicalRepresentation Component (which includes all incoming, dynamic data) tomake its determination concerning the existence of a chemical orbiological cloud and/or the motion of a cloud. The Agent Interface 301is responsible for gathering data from the hierarchy or blackboard andproviding the information to the PRC and Mission Knowledge Base and isalso responsible for placing information into the hierarchy or placingit onto the blackboard after it is output from the Cognitive Component.

The architecture for each agent in the invention consists of a reasoningmechanism, specialized knowledge, communication facilities, andknowledge base access methods. Each agent can be, in turn, composed ofmultiple agents so that the complexity of a given agent's operation canbe hidden from all other agents and the communication interface betweenagents remains the same regardless of the internal complexity of theagent. FIG. 4 contains a detailed representation of the architecture ofa single intelligent agent. As shown in this FIG., each agent's decisionengine has six main components, components which are themselves agentsand form an agent society, a Current World Model 401, an Expected WorldModel 405, an Environment Analyst 402, an Other Sensor Inputs Analyst403, a Feedback Analyst 404, and an Engine Memory 406. All six of thesecomponents are able to send and receive data and analyses from any othercomponent of a decision engine. Within a decision engine, the CurrentWorld Model component contains a description of the state of theenvironment that has been assembled by the agent based upon its inputsderived from its sensors. This current description is compared againstthe Expected World Model component to detect changes in the world andunexpected events.

By way of explanation, the term “world model”, recited here and inconnections with FIG. 7 and FIG. 8 of the drawings for examples, is anartificial intelligence term and refers to the computer-stored andmaintained representation of the contents of the volume within asensor's field of view and by extension to the field of view for all ofthe sensors for the present invention system. No model of the world as aglobe is intended or maintained. The expected world model is the agent'sestimate of the anticipated state of the volume within the sensor'sfield of view.

The Environment Analyst component uses information contained in theworld models and information provided by the Other Sensor Inputs Analystcomponent to determine if a cloud (as indicated by a segment withcertain properties) exists in the environment and to track a cloud if itdoes exist based upon the individual intelligent agent's inputs. TheEnvironment Analyst also generates feedback to be sent to other agentsin the agent hierarchy in the invention. Feedback can consist ofinformation such as there are too many segments (or chemical orbiological cloud plumes) being provided, do not forward raw sensor data,increase the number of segments, or any other performance factors. Theoutput from the Environment Analyst that is sent to other intelligentagents is written in XML according to the DTD presented below. The OtherSensor Inputs Analyst component is responsible for accepting dataprovided by other intelligent agents and for using it to help theEnvironment Analyst component to determine if a chemical or biologicalcloud exists and to enable the tracking of a chemical or biologicalcloud.

The Feedback Analyst component takes information provided as feedbackfrom other intelligent agents in the hierarchy and provides it to theother components in its decision engine except for the Current WorldModel. The Feedback Analyst filters the feedback to extract the feedbackappropriate to its particular engine and also to its current state. Thefeedback allows the decision-making components of an engine to refinetheir operation based upon the perceived utility of their outputs byother intelligent agents in the invention. The information that comesinto the Feedback Analyst is written in XML according to DTD presentedbelow. The Engine Memory component maintains a record of all of thedecisions and outputs of the other six components as well as a record ofworld models so that they can draw upon the memory to make futuredecisions. The memory holds information like the number of segmentsdetected for each type of segmentation performed, time-stamped feedback,location of correlations that were detected, prior weather conditionsand significant aspects of prior world models formed by the agent.

Several factors support our decision to use XML for communicationbetween intelligent agents in the invention. First, XML is a flexibleapproach to formatting. The XML capability to define and use custom tagsand the minimal requirements imposed by the language permit expressionof the transmission format robustly within the boundaries of thelanguage. Second, XML is widely used and is standardized; therefore, thebasic components of the language are stable and well understood. Third,XML is precise, it has a well defined set of rules for describing adocument and for ordering the contents of a document but does notspecify semantics. As a result, XML provides the basis for developing acommon data format that is robust in the face of data corruption,self-describing in terms of tag meaning, and extendable to accommodateunforeseen data requirements. Fourth, because XML supports thedefinition of custom tag-sets and custom structures that are completelycontained within the document, an XML-based specification can beautomatically searched and categorized by computer programs instead ofmanually. Finally, XML supports the creation and use of multipart,distributed documents and supports interchange of data between agentsand agent societies. Additional information relating to the XMLcommunication is included in the appendix to this specification. Eachagent's output is contained within a Document Type Definition (DTD) forthe invention. The DTD for all of the agents in the invention ispresented in Table 2 below. This DTD is used by each agent tocommunicate with other agents in the hierarchy and between agents in asociety. Vocabulary tags and assigned meanings for certain intelligentagent outputs in Table 2 appear in Table 3 also shown below.

TABLE 2 XML Document Type Definition <?XML version=“1.0”encoding=“UTF-8”?> <!DOCTYPE INTELLIGENT_AGENT_OUTPUT SYSTEM [ <!ELEMENTINTELLIGENT_AGENT_OUTPUT > <!ELEMENT Attack occurring (#PCDATA)><!ELEMENT Attack concluded (#PCDATA)> <!ELEMENT Number of sensors(#PCDATA)> <!ELEMENT Sensor type (#PCDATA)> <!ELEMENT Raw sensor output(#PCDATA)> <!ELEMENT Boundary for sensor footprint on (#PCDATA)> surface<!ELEMENT Sensor frequency range (#PCDATA)> <!ELEMENT Sensitivity ofsensor (#PCDATA)> <!ELEMENT Resolution of sensor (#PCDATA)> <!ELEMENTSensor altitude (#PCDATA)> <!ELEMENT Sensor position (#PCDATA)><!ELEMENT Orientation of sensor (#PCDATA)> <!ELEMENT Raw sensor output(#PCDATA)> <!ELEMENT Composite assessment (#PCDATA)> <!ELEMENT Compositeassessment confidence level (#PCDATA)> <!ELEMENT Number of decisionengine assessments (#PCDATA)> <!ELEMENT Decision engine assessment(#PCDATA)> <!ELEMENT Decision engine assessment confidence (#PCDATA)>level <!ELEMENT Number of input assessments (#PCDATA)> <!ELEMENT Inputassessment value (#PCDATA)> <!ELEMENT Input assessment confidence(#PCDATA)> <!ELEMENT Feedback (#PCDATA)> <!ELEMENT Number of clouds(#PCDATA)> <!ELEMENT Cloud identifier (#PCDATA)> <!ELEMENT Cloudconfidence (#PCDATA)> <!ELEMENT Cloud location (#PCDATA)> <!ELEMENTCloud altitude (#PCDATA)> <!ELEMENT Main axis (#PCDATA)> <!ELEMENT Minoraxis (#PCDATA)> <!ELEMENT Change in main axis (#PCDATA)> <!ELEMENTChange in minor axis (#PCDATA)> <!ELEMENT Cloud mean height (#PCDATA)><!ELEMENT Cloud maximum height (#PCDATA)> <!ELEMENT Change in cloud meanheight (#PCDATA)> <!ELEMENT Cloud direction of motion (#PCDATA)><!ELEMENT Cloud velocity of motion (#PCDATA)> <!ELEMENT Millimeter wavepenetration (#PCDATA)> <!ELEMENT Average infrared emission for the cloud(#PCDATA)> <!ELEMENT Cloud reflectance (#PCDATA)> <!ELEMENT Cloudhumidity (#PCDATA)> <!ELEMENT Cloud temperature (#PCDATA)> <!ELEMENTNumber of plumes (#PCDATA)> <!ELEMENT Plume identifier (#PCDATA)><!ELEMENT Plume confidence (#PCDATA)> <!ELEMENT Plume location(#PCDATA)> <!ELEMENT Plume altitude (#PCDATA)> <!ELEMENT Main axis(#PCDATA)> <!ELEMENT Minor axis (#PCDATA)> <!ELEMENT Change in main axis(#PCDATA)> <!ELEMENT Change in minor axis (#PCDATA)> <!ELEMENT Plumemean height (#PCDATA)> <!ELEMENT Plume maximum height (#PCDATA)><!ELEMENT Change in plume mean height (#PCDATA)> <!ELEMENT Direction ofmotion (#PCDATA)> <!ELEMENT Velocity of motion (#PCDATA)> <!ELEMENTPlume reflectance (#PCDATA)> <!ELEMENT Plume humidity (#PCDATA)><!ELEMENT Plume temperature (#PCDATA)> <!ELEMENT Ambient humidity(#PCDATA)> <!ELEMENT Ambient air temperature (#PCDATA)> <!ELEMENTNatural clouds present (#PCDATA)> <!ELEMENT Surface moisture (#PCDATA)><!ELEMENT Surface temperature (#PCDATA)> <!ELEMENT Sun position(#PCDATA)> <!ELEMENT Moon position (#PCDATA)> <!ELEMENT Increase numberof segments (#PCDATA)> <!ELEMENT Decrease number of segments (#PCDATA)><!ELEMENT Forward all raw sensor data (#PCDATA)> <!ELEMENT Do notforward raw sensor data (#PCDATA)> <!ELEMENT Wind direction (#PCDATA)><!ELEMENT Wind velocity (#PCDATA)> <!ELEMENT Elapsed time since eventbegan (#PCDATA)> <!ELEMENT Number of natural clouds (#PCDATA)> <!ELEMENTNatural cloud id (#PCDATA)> <!ELEMENT Natural cloud location (#PCDATA)><!ELEMENT Natural cloud altitude (#PCDATA)> <!ELEMENT Natural Cloud Mainaxis (#PCDATA)> <!ELEMENT Natural Cloud Minor axis (#PCDATA)> <!ELEMENTNatural Cloud assessment confidence (#PCDATA)> > </INTELLIGENT_AGENT_OUTPUT>

Table 3 provides the definition for each tag used in the DTD for theinvention.

TABLE 3 Vocabulary Tags and Assigned Meanings for Certain IntelligentAgent Outputs TAG MEANING Attack occurring Signal that an attack hasbeen detected, nominal value is false Attack concluded Signal thatattack has concluded, signal to reset all timers and begin scanning fornew attack, nominal value is true Number of sensors Number of sensorsused to make the assessment reported in the current report Sensor typeType of sensor used to provide an assessment input Boundary for sensorfootprint on Latitude and longitude of each corner the surface of therectangle for the sensor foot- print or the latitude and longitude ofthe center of the circle of the foot- print and the circle's radius;defines the boundary of the world model volume for a sensor Sensorfrequency range Operational range of the sensor, from lowest effectivefrequency to highest effective frequency used by the sensor for thisreport. Resolution of sensor In square meters Sensor altitude Given inmeters above mean sea level Sensor position Given in right ascension anddeclination Orientation of sensor Relative to prime meridian and equatorRaw sensor output Values produced by a sensor Composite assessmentOverall assessment by the intelligent agent that whether a chemical orbiological agent plume is present Composite assessment confidenceConfidence level of the intelligent level agent that its assessment ofthe presence of a chemical or biological agent plume is correct Numberof decision engine Number of assessments based on the assessmentscomputations of a single decision engine that a chemical or biologicalattack is occurring included in this report Decision engine assessmentAssessment of the decision engine whether a chemical or biological agentplume is present Decision Engine assessment Confidence value for theassessment confidence level of the decision engine whether a chemical ofbiological agent plume is present Number of input assessments Number ofinput assessments used by the intelligent agent to make its computationsInput assessment value Value of an input assessment used by theintelligent agent to make its computations Input assessment confidenceConfidence value attached to the value of an input assessment used bythe intelligent agent to make its computations Feedback Tag to indicatethe presence of feedback from a higher level in the hierarchy Number ofclouds Number of clouds that have not been identified as either achemical or biological plume or natural cloud in this report Cloudidentifier Unique identifier assigned to a cloud by an intelligent agentCloud confidence Confidence level value that a cloud was detected Cloudlocation Latitude and longitude of the cloud Cloud altitude Given inmeters above mean sea level Main axis The long axis of the plume Minoraxis The short axis of the plume Change in main axis Change in length ofmain axis since last report Change in minor axis Change in length ofminor axis since last report Cloud mean height Mean height of the cloud,in meters above the surface of the Earth Cloud maximum height Maximumheight of the cloud, in meters above the surface of the Earth Change incloud mean height Change in mean height of the cloud since last reportCloud direction of motion Direction of the movement of the center of thecloud relative to true north Cloud velocity of motion Velocity of thecenter of the cloud in kilometers per hour Millimeter wave penetrationWhether a radar can penetrate the cloud to the ground Average infraredemission for the cloud Cloud reflectance The ratio of the amount ofelectro- magnetic radiation that reflects off the surface of the cloudto the amount of radiation that strikes the cloud Cloud humidity Givenas percent relative humidity Cloud temperature Given in degreescentigrade Number of plumes Number of chemical and biological plumesreported in this report Plume identifier Unique identifier assigned to aplume by an intelligent agent Plume confidence Confidence value that theplume assessment is correct Plume location Latitude and longitude of thecenter of the plume. Plume altitude Given in meters above mean sea levelMain axis The long axis of the plume Minor axis The short axis of theplume Change in main axis Change in length of main axis since lastreport Change in minor axis Change in length of minor axis since lastreport Plume mean height Mean height of the plume, in meters above thesurface of the Earth Plume maximum height Maximum height of the plume,in meters above the surface of the Earth Change in plume mean heightChange in mean height of the plume since last report Direction of motionDirection of the movement of the center of the plume relative to truenorth Velocity of motion Velocity of the center of the plume inkilometers per hour Plume reflectance The ratio of the amount ofelectro- magnetic radiation that reflects off the surface of the cloudto the amount of radiation that strikes the cloud Plume humidity Givenas percent relative humidity Plume temperature Temperature of a plumegiven in degrees centigrade Ambient humidity Given as percent relativehumidity Ambient air temperature Temperature of the air, given indegrees centigrade Natural clouds present Whether or not clouds arepresent in the atmosphere in the same area that a sensor is watching,given as a true or false value Surface moisture Amount of moisturemeasured on the surface Surface temperature Temperature of Earth'ssurface Sun position Given in right ascension and declination Moonposition Given in right ascension and declination Increase number ofsegments Directive to a lower level agent to increase the number ofsegments that it identifies, usually accomplished by using some form offiner grained segmentation values Decrease number of segments Directiveto a lower level agent to decrease the number of segments that itidentifies, usually accomplished by using some form of coarser grainedsegmentation values Forward all raw sensor data Directive to a lowerlevel agent to place the raw sensor values into the intelligent agenthierarchy. Do not forward all raw sensor data Directive to a lower levelagent to stop placing the raw sensor values into the intelligent agenthierarchy. Wind direction Direction the wind is coming from, given inrelation to true north Wind velocity Velocity of the wind, given inkilometers/hour Elapsed time since event began Time in seconds since theintelligent agent first detected a chemical or biological agent plumeNumber of natural clouds Total number of natural clouds reported in thisreport Natural cloud id Unique identifier assigned to a natural cloud byan intelligent agent Natural cloud location Latitude and longitude ofthe center of the natural cloud. Natural cloud altitude Given in metersabove mean sea level Natural Cloud Main axis The long axis of thenatural cloud Natural Cloud Minor axis The short axis of the naturalcloud Natural Cloud assessment Confidence value that the naturalconfidence cloud assessment is correct

FIG. 5 in the drawings shows the basic flow for the processing of datawithin the invention among all of the agents in a single agent society.The FIG. 5 agent society is composed of agents of the FIG. 4 type havinga common tasking to identify and/or track one or more different types ofchemical or biological clouds. The FIG. 5 agent society operates asfollows. Communication between agents is accomplished using XML and datasegmentation is accomplished with the use of one or more standard imagesegmentation techniques. At step 500 in FIG. 5 the data from the sensorsconnected to the agent under discussion is acquired and at 510 theclouds within the view range of the sensors are extracted using thesegmentation information at 505 and information from the agent knowledgebase 512. The knowledge base information is stored in the agent andcomes from data gathered from subject matter experts. Similar storageand sourcing is used for the other knowledge bases in FIG. 5 throughFIG. 10 herein. At 520 in FIG. 5, using information from the knowledgebase 522, parameters, and data about each cloud is determined; thisinformation includes data such as location, size, velocity, density, andrate of change of expansion of the cloud. At 530, the features for eachcloud are correlated within the cloud to insure they are consistentusing information from the current world model at 535 and information inthe knowledge base. At 540, the information for each cloud is correlatedbetween all of the agents using information in the knowledge base. At550, the description for each cloud is output in XML to other agentsand/or to the later discussed blackboard. This information is also sentto the engine memory at 570 and the feedback analyst at 560. Theinformation in the engine memory at 570 and the feedback analyst at 560is sent to the expected world model where it is used in conjunction withthe knowledge base to revise the expected world model at 580 and toupdate the current world model at 535. The information from the feedbackanalyst at 560 is also sent to the other sensor input analyst at 590where it is used in conjunction with the information in the knowledgebase to determine changes to be made to the knowledge base data used byother agenda to identify clouds and determine their parameters. The dataoutput from 560 is also used to adjust the segmentation thresholds usedat 505 and the content of the knowledge base. Processing continues aslong as the sensors continue to supply input data.

FIG. 6, which is made up of the parts FIG. 6A and FIG. 6B, shows theoperation of the Environment Analyst intelligent agent. The EnvironmentAnalyst uses information contained in the current world model,information provided by sensors, and information provided by the OtherSensor Inputs Analyst to determine if a cloud (as indicated by a segmentwith certain properties) exists in the environment and also to track acloud. The Environment Analyst also generates feedback sent to otheragents in the agent hierarchy and in its society in the invention.Feedback can consist of information such as there are too many segments(or chemical or biological cloud plumes) being detected or provided, donot forward raw sensor data, increase the number of segments, or anyother performance factors, parameters, or directives. The output fromthe Environment Analyst is sent to other intelligent agents using XML.The Environment Analyst operates as follows. Sensor data ia provided at600 and is sent to the search function at 610 where the data issegmented to extract clouds using information in the knowledge base anddata from other sensors and their analysis as provided at 601. Theresults of the segmentation process is sent to the decision at 620 wherethe agent determines if one or more clouds have been located usinginformation in the knowledge base. If no clouds are identified,processing return to step 600. If clouds are located, processingproceeds to 630 where information from the knowledge base is used todetermine if there is a volume to be examined. If there is not a volume,processing returns to 600, otherwise processing proceeds to thedetermination at 640 where information from the knowledge base is usedto determine the characteristics of the volume. Processing then goes onto the correlation at 650 where information from the knowledge base isused to correlate information to determine the characteristics for eachcloud.

Processing then advances to step 660 where information from theknowledge base and the current world model is used to determine if thecloud is a chemical or biological cloud. If it is determined that thecloud is not a chemical or biological cloud, processing returns to 630;otherwise processing advances to the step 665 where the data for thecloud is sent to the Feedback agent. In step 670, the EnvironmentAnalyst uses information from the current world model and knowledge baseto determine if the cloud is in the current world model; if not thenprocessing goes to the function at 685. At 685, a new cloud identifieris created along with its parameters and processing advances to 686where the description of the cloud and its data is output in XML andsent to the other agents; processing then goes to 683 for modelupdating. If at 670 it is determined that the cloud is in the currentworld model, processing advances to 680 where the cloud and its data ismatched to the cloud in the current world model and at 683 the expectedworld model is updated. Processing then advances to 683 where thecurrent world model and other models are updated and then to 690 wherethe engine memory is updated. Processing then returns to 630 forrepetitions until no more volumes remain to be examined.

FIG. 7 in the drawing shows the operation of the Current World Modelagent. Within a decision engine, the Current World Model is anintelligent agent and contains a description of the state of theenvironment that is assembled by the agent based upon data from itssensors and other agents. The FIG. 7 Current World Model agent operatesas follows. At 700, the agent retrieves the latest Environment Analyst'soutput data. This information and the Expected World Model at 705 areused at 710 to make two lists, one list composed of clouds in theenvironment analyst's output and the other list composed of clouds inthe expected world model's output. At 720, the cloud at the top of theenvironment analyst's output list is compared against all of the cloudsin the list formed from the output from the expected world model usinginformation in the knowledge base to aid in the comparison. At 730, if amatch is not found for a cloud in the two lists using informationcontained in the knowledge base then a new cloud has been located andtherefore processing advances to 735. At 735, the new cloud and itsparameters are added to the current world model and processing advancesto 737. If, at 730 a match is found for a cloud then processing proceedsto 740 where it is determined, using information in the knowledge base,if the cloud appears in both lists but the parameters for the cloud havea difference.

If a 740 the cloud appears in both lists but there is a difference inparameters then processing advances to 745; at 745 the cloud'sparameters are resolved using information in the knowledge base and thenthe cloud's parameters are updated in the current world model using theresolved parameter information. From 745, processing then advances to737. At 737, the cloud and its parameters are sent to the FeedbackAnalyst agent and Expected World Model agent. From 737, processingadvances to 750. If at 740 there is no difference in the cloud'sparameters; which means that the dead reckoned cloud and its parametervalues computed for the expected world model agent match the valuesobserved in the real world by the environment analyst, then the currentworld model agent has found a complete match and no changes need to bemade to the current world model since it is accurate; thereforeprocessing advances to 750, where the cloud and all of its parameterinformation are sent to the engine memory, and the top cloud (the onethat was just examined) is removed from the environment analyst outputlist. Processing then advances to 760, where it is determined whetherclouds remain in the environment analyst's output list. If cloudsremain, then processing returns to 720 where the next cloud on the listis examined. If clouds do not remain then processing of the latestoutput from the environment analyst is complete so processing returns to700 where the next set of output data from the environment analyst isobtained.

FIG. 8 show the operation of the Extended World Model agent. Within adecision engine, the Expected World Model is maintained by anintelligent agent and contains a description of the state of theenvironment that has been computed by the agent based upon its deadreckoning of cloud parameters based upon inputs derived from its sensorsand from other agents. In the FIG. 8 agent, the Expected World Model andCurrent World model are compared to detect differences and to detectunexpected events. The Expected World Model agent operates as follows.At 805, the agent polls for data from the Environment Analyst, Enginememory, the Other sensor inputs analyst, and feedback at 801, 802, 803,and 804 respectively. Once new data is received, processing advances to810 where the agent determines if there is new cloud data by using theinformation received and information in the knowledge base. If no newcloud data is received, processing advances to 870 where the agent deadreckons the size, the position, and all other parameters for all itsclouds and sends an update to the engine memory. If there is new clouddata at 810, processing advances to 820 where the agent uses informationin the knowledge base to determine if the new cloud data corresponds toany cloud in the expected world model.

If the cloud and its data do not correspond to any cloud in the ExpectedWorld Model, then processing proceeds to 830 where the expected worldmodel is updated and processing then proceeds to 835 where the cloudinformation is sent to the feedback agent. If the cloud and its data docorrespond to data in the expected world model processing proceeds to840 where the agent uses information in its knowledge base to determineif there is a difference between the new data for the cloud and the datafor the cloud already in the expected world model. If there is nodifference, processing advances to 870. If there is a difference,processing advances to 860 where the agent uses information in itsknowledge base and the cloud's new data to revise the cloud's parametersin the expected world model. From 860, processing in the agent proceedsto 880 where the differences between the new parameters and theparameters in the expected world model are sent to the feedback agent inXML format. From 880, processing advances to 870. From 870 processingadvances to 805 where the agent polls for new data.

FIG. 9 shows the operation of the Other Sensors Input Analyst. The OtherSensor Inputs Analyst is an intelligent agent and is responsible foraccepting data provided by other intelligent agents and the enginememory and for using the data to help the Environment Analyst componentto determine if a chemical or biological cloud exists, to detect errorsin the world model, and to enable the tracking of a chemical orbiological cloud. The Other Sensor Inputs Analyst agent operates asfollows. At 905, the agent polls for new data from other agents, i.e.,from Engine memory, the expected world model, the feedback analyst, andthe environment analyst at 901, 902, 903, 904, and 906 respectively.Once new data is received, processing advances to 910 where the agentmakes two lists, one containing all of the data from the other agentsand the other list containing all of the data in the expected worldmodel. Processing then advances to 920, where the agent takes the cloudat the top of the expected world model and determinate if this cloudappears in the other list. If the cloud appears in both lists,processing advances to 930, otherwise processing advances to 940. At930, the agent uses the information in its knowledge base to determineif there is a significant difference in the information about the cloudon the two lists. If there is a significant difference at 930,processing advances to 933, where the agent uses information in itsknowledge base to reconcile the differences and sends the informationabout the reconciliation to the feedback agent; processing then advancesto 935 where the agent updates the current and expected world modelswith the reconciled data for the cloud and then processing advances to950.

If at 930 there is not a significant difference in cloud data thenprocessing advances to 940, where the agent removes the cloud and itsinformation from the top of the expected world model list. Processingthen advances to 950, where the agent determines if an entry remains inthe expected world model list. If an entry remains in the list, cloudsremain to be processed and so processing returns to 920. If at 950 anentry does not remain in the Expected World Model list, then processingadvances to 960, where the agent determines if data from the environmentanalyst and expected world model have been analyzed. If they have, thenprocessing returns to 905 where the agent polls for new data. If at 960it is determined that data from the environment analyst and expectedworld model have not been analyzed, then the environment analyst datahas not been examined, so processing advances to 980. At 980, the agentmakes two lists, one containing all of the data from the environmentanalyst and the other list containing all of the data in the expectedworld model; processing then returns to 920 where processing continueson the most recent data received from the environment analyst.

FIG. 10, which is made-up of the parts FIG. 10A and FIG. 10B, showsoperation of the Feedback Analyst. The Feedback Analyst agent takesinformation provided as feedback from other intelligent agents in thehierarchy and from local agents in the agent society, the engine memory,and the Expected World Model and provides feedback concerningperformance accuracy to the other agents in its agent society. TheFeedback Analyst agent operates as follows. At 1005, the agent polls fornew data from other agents, Engine memory, and the expected world modelat 1001, 1002, and 1003 respectively. Once new data is obtained at 1005,processing advances to 1010 where the agent determines if it has beenprovided with any directives to change thresholds or data; if it hasreceived directives processing advances to 1012 otherwise processingadvances to 1018. At 1012 the agent determines if the directiveconflicts with any other directives it may have received; if there is aconflict, processing advances to 1013, otherwise processing advances to1014. At 1013, the agent uses its knowledge base to resolve the conflictin the directives and then processing advances to 1014. At 1014, theagent changes the data and/or thresholds as ordered in all of thedirectives it currently has received and then processing advances to1018.

At 1018, the agent removes all of the directives from the input data andthen organizes all remaining items into a list ordered by cloudidentifier; processing then advances to 1020. At 1020, the agentexamines the cloud and its data at the top of the list and determines ifthis cloud corresponds to any clouds currently known to this agent. Ifthe cloud does not, processing advances to 1022 otherwise processingproceeds to 1030. At 1022, the Feedback Analyst determines the otheragents that need the data associated with this newly identified cloudand at 1023 sends the data to the agents that require the data;processing then advances to 1024. At 1024 the Feedback Analyst agentuses its knowledge base to determine if there is data that is mustchange in order to allow it to detect clouds of this type in the futurebecause a potential error in the knowledge base has been detected. Ifdata must be changed, processing advances to 1025 where the agent usesinformation in the knowledge base to appropriately change theparameters, thresholds, weights, or other properties associated withdetecting this cloud and then processing advances to 1075.

If no data needs to be changed, then processing advances to 1077. If at1020 the cloud at the top of the list does correspond to a cloud knownto this agent, processing advances to 1030, where the agent uses theinformation in its knowledge base to determine if the cloud correspondsto any geographical area covered or examined by this agent. If it does,then processing advances to 1022; otherwise processing proceeds to 1035.At 1035, the agent uses the information in its knowledge base todetermine if the top entry in the list contains data for a cloudproperty, if it does then processing proceeds to 1022; otherwiseprocessing advances to 1040. At 1040, the agent uses the information inits knowledge base to determine if the top entry in the list containsdata that corresponds to or affects a threshold value used by thisagent; if it does then processing advances to 1022 otherwise processingadvances to 1050. At 1050, using the knowledge base the agent determinesif the entry at the top of the list makes use of a parameter value forclouds that the agent uses, if it does then processing proceeds to 1022otherwise processing advances to 1052. At 1052, the agent makes a listof all of the properties reported for the cloud at the top of the listand then processing advances to 1054. At 1054, the agent uses theinformation in the knowledge base to determine if all of the propertiesfor the cloud correlate with each other, to insure that there are noinconsistencies in the properties. If the parameters correlate,processing advances to 1055 otherwise processing proceeds to 1065. At1055, the Feedback agent sends a directive message to source agents usedto identify the cloud to increase their selection weights and confidencevalue for parameters used to identify the cloud since these parameterswere able in combination to detect the cloud; and then processingadvances to 1060. If at 1054 the properties for the cloud do notcorrelate, then processing advances to 1065.

At 1065, the Feedback agent uses the information in its knowledge baseto resolve the conflicts in the properties and then processing advancesto 1067. At 1067, the Feedback agent sends a directive message to thesource agents for the cloud to inform them of property value adjustmentsthat have been made. Processing then advances to 1070, where theFeedback agent uses the information in the knowledge base to adjust theconfidence values for the properties for the cloud. Processing thenadvances to 1075, where the agent sends a directive message to sourceagents for the cloud to inform them of the adjustments that were made tothe confidence values. Processing then advances to 1060, where the agentsends the cloud and parameter values to the blackboard and/or to anyagents at higher levels in the agent hierarchy. From 1060, processingadvances to 1077, where the Feedback agent removes the cloud and all ofits parameters from the top of the list and processing advances to 1080.At 1080, the Feedback agent determines if any clouds remain on the list.If clouds remain in the list, processing returns to 1020, otherwiseprocessing proceeds to 1005 where the Feedback agent polls for new dataonce again.

In summary, substantial benefits have been made available throughutilizing the principles of the present invention; these benefitsinclude:

-   -   1. Complete coverage for detection and tracking purposes across        a wide area.    -   2. The sensitivity and accuracy of the system is only limited by        the characteristics of the sensors and the contents of the        knowledge base.    -   3. Capability for real-time monitoring of large-scale substance        attacks anywhere within the United States or the world.    -   4. While each data source employed is known, it is their        combination and the sensor resolution possible that makes the        invention notable. It is believed that no one has attempted        performing the type of real-time data acquisition, synthesis,        and fusion needed to detect and track agents in the manner of        the invention.    -   5. The intelligent agent hierarchy can be made as complex as        required.    -   6. The intelligent agent hierarchy is scalable in terms of        computational power and number of parameters to be examined. Any        number of computers can be used in the invention, including up        to one computer per intelligent agent.    -   7. The intelligent agent hierarchy can be scaled to examine any        desired number of parameters or factors.    -   8. The intelligent agent hierarchy can be scaled to use the data        produced by any desired number of sensors.    -   9. The intelligent agent hierarchy and each intelligent agent        can use any decision making/analysis system.    -   10. The intelligent agent hierarchy can use multiple different        decision making/analysis systems throughout the hierarchy.    -   11. Each agent in the intelligent agent hierarchy operates        independently.    -   12. The included data transmission format provides an        application, reasoning system, reasoning format, and knowledge        content independent means for representing knowledge.    -   13. Any intelligent agent can use the included data transmission        format to transmit the results of its analysis to any other        intelligent agent in the invention's system.    -   14. The data transmission format can be used to transmit data        between intelligent agents using a computer network or any type        of data storage medium.    -   15. The data transmission format is completely self-contained,        documentation external to the information in the format is not        required to understand the syntax and semantics of the        information contained in the information transmission format.    -   16. The data transmission format can be directly read and used        by either a human or a computer.    -   17. The data transmission format permits the security        classification of each element of a knowledge base to be        embedded within the knowledge base.    -   18. The data transmission format permits the accuracy,        assurance, and non-corruption of the contents of a transmission        to be insured and verified using standard computer checksum and        encryption technologies.    -   19. The data transmission format inherently supports multiple        levels of security within a transmission and inherently support        automatic, computer-controlled selective extraction of data        within a transmission based upon each element's individual        classification.    -   20. The data transmission format inherently supports automatic        verification of the propriety, appropriateness, or suitability        of a data transfer on an element-by-element basis.    -   21. The data transmission format provides inherent support for        documenting the pedigree, source authority, and augmentation of        the contents of a transmission for each transmission.

The foregoing description of the preferred embodiment has been presentedfor purposes of illustration and description. It is not intended to beexhaustive or to limit the invention to the precise form disclosed.Obvious modifications or variations are possible in light of the aboveteachings. The embodiment was chosen and described to provide the bestillustration of the principles of the invention and its practicalapplication to thereby enable one of ordinary skill in the art toutilize the inventions in various embodiments and with variousmodifications as are suited to the particular scope of the invention asdetermined by the appended claims when interpreted in accordance withthe breadth to which they are fairly, legally and equitably entitled.

APPENDIX 1 The eXtensible Markup Language

The eXtensible Markup Language (XML) is a meta-language permitting theuser/designer to define components of the language (syntax, data types,vocabulary, and operators) needed to achieve a required capability usinga Document Type Definition (DTD). An XML-based transmissionrepresentation has three parts, the XML document, the Document TypeDefinition, and the XML Stylesheet. The stylesheet adresses onlyappearance issues, and therefore is not further discussed. An XML-baseddocument has a standard format and a limited number of pre-defined tags.An XML document consists of one or more elements, marked as shown in theExample 1 below.

Example 1: XML Document Element Format <Start> Element text to betransmitted </Start>

An element consists of two tags, an opening tag (< >) and a closing tag(</>), with the name of the tag appearing between the < and the >. Thenesting of the tags within the XML document must correspond to thenesting specified within the DTD. An XML document always begins with theXML declaration in the following format, <?xml version=“1.0”standalone=“no”?>. The declaration specifies the version of XML beingused and whether or not an external DTD is required. This particularspecification indicates that an external DTD is needed by a parser tounderstand the structure of the document; alternatively, saying yesmeans that a DTD is embedded within the XML document. The second linecontains the entry: <?DOCTYPE DMTITE:RuleBased SYSTEM “sample.dtd”>.This entry defines the root element of the document (DMTITE:RuleBased)and the location of the DTD. The remainder of the document contains tagsdefined in the DTD and the corresponding data to be transmitted.

The Document Type Definition (DTD) specifies how the elements (contents)of a document must relate to each other. Each element is defined using a<!ELEMENT> declaration in the following format: <!ELEMENT elementnamerule >. There are a number of keywords that can be used to define therule, including the keywords ANY and #PCDATA. The ANY keyword indicatesthat any character data or keyword can appear within the element. Thekeyword #PCDATA indicates that only generally character data can appearwithin the element. To indicate that multiple elements must appear in aspecific sequence, commas are used to separate the instances.Parentheses are used to group declarations. Within a declaration, one ofthree operators can be used to specify the number of occurrences of anelement. The ? operator indicates that the element must appear once ornot at all. The + operator indicates that the element must appear atleast once. The * operator indicates that the element can appear anynumber of times or not at all. Within the DTD, attributes can be definedfor any element. An attribute is defined using the <!ATTLIST>declaration, which has the form <!ATTLIST target_element attribute_nameattribute_type default>. Data types include character data (CDATA) andenumerated data, as well as the keywords #REQUIRED, #IMPLIED, and#FIXED. Using these components as a basis in conjunction with a set oftopic-specific keywords, a document type can be defined. The unusualcapitalization of the word “eXtensible” employed herein is in accordancewith prevailing practice in the artificial intelligence art. Herein thisword is intended to be of a generic description rather than a specificlanguage limiting nature.

1. The method of detecting and tracking chemical and biological warfareagent plumes over a geographic area, said method comprising the stepsof: disposing a plurality of diverse input condition responsive warfareagent sensor elements above said geographic area of detecting andtracking; coupling an output from each of said diverse input conditionresponsive warfare agent sensor elements to a sensor-specific firstlevel intelligent agent decision element in a hierarchical array ofintelligent agent decision elements; extracting, in said sensor specificfirst level intelligent agent decision element, cloud-related signalindicia from said output, said extracting including applying segmentedoutput data and first level intelligent agent knowledge base data to adecision engine algorithm portion of said sensor-specific first levelintelligent agent decision element to generate a sensor-specific firstlevel intelligent agent decision element eXtensible markup languagedocument type definition output; said sensor-specific first levelintelligent agent decision element eXtensible markup language documenttype definition output also including a cloud identification output anda warfare agent sensor element output quality-determined firstconfidence signal; applying said eXtensible markup language documenttype definition output of said sensor-specific first level intelligentagent decision element to a plurality of second and successive other midlevel intelligent agent decision elements in said hierarchical array ofintelligent agent decision elements; evaluating in said mid levelintelligent agent elements cloud, weather condition, ground conditionand cloud duplicate related outputs received via said mid leveleXtensible markup language document type definition outputs of saidfirst level intelligent agent elements; said evaluating step including adetermination of possible natural cause for said cloud-related outputs,a determination of said cloud-related outputs being a warfare agentplume and a determination of a second confidence signal, saiddeterminations being encoded into an intelligent agent decision elementeXtensible markup language document type definition output; connectingsaid second intelligent agent decision element eXtensible markuplanguage document type definition output to a plurality of upper levelintelligent agent decision elements in said hierarchical array ofintelligent agent decision elements; and determining in said pluralityof upper level intelligent agent decision elements in said hierarchicalarray of intelligent agent decision elements an occurrence of a warfaresubstance attack event and a course of travel for said warfare agentplume in said geographic area, said determining including determining athird confidence signal level and encoding of said determining stepsignals into an intelligent agent decision element eXtensible markuplanguage document type definition output.
 2. The method of detecting andtracking chemical and biological warfare agent plumes over a geographicarea of claim 1 wherein said diverse input condition responsive warfareagent sensor elements include at least one of an infrared radarapparatus, a synthetic aperture radar apparatus and an ultravioletsensor apparatus.
 3. The method of detecting and tracking chemical andbiological warfare agent plumes over a geographic area of claim 1wherein said intelligent agent decision elements each comprise aphysical representation component, a cognitive component, an agentinterface component and a knowledge base component.
 4. The method ofdetecting and tracking chemical and biological warfare agent plumes overa geographical area of claim 3 wherein said intelligent agent decisionelement physical representation component comprises an externalenvironmental model inclusive of terrain description data, sun positiondata, moon position data, major water body description data, weatherdata, soil temperature data, soil moisture content data and cloudreflectance data.
 5. The method of detecting and tracking chemical andbiological warfare agent plumes over a geographical area of claim 3wherein said intelligent agent decision element knowledge base componentincludes mission related input and output data definitions, intelligentagent decision element instructional data and a confidence determinationalgorithm.
 6. The method of detecting and tracking chemical andbiological warfare agent plumes over a geographic area of claim 3wherein said intelligent agent decision element cognitive componentincludes a knowledge base component and a physical representationcomponent responsive decision engine element.
 7. The method of detectingand tracking chemical and biological warfare agent plumes over ageographic area of claim 6 wherein said knowledge base component andphysical representation component responsive decision engine elementincludes a current world model processing algorithm, an expected worldmodel processing algorithm, an environment analyst processing algorithm,an other sensor inputs analyst processing algorithm, a feedback analystprocessing algorithm and an engine memory component each having two waycommunication within said decision engine element.
 8. The method ofdetecting and tracking chemical and biological warfare agent plumes overa geographic area of claim 7 wherein said environment analyst processingalgorithm includes a cloud-related sensor data examination routine. 9.The method of detecting and tracking chemical and biological warfareagent plumes over a geographic area of claim 7 wherein said currentworld model processing algorithm includes a cloud-related examinationroutine having environment analyst processing algorithm generated inputdata.
 10. The method of detecting and tracking chemical and biologicalwarfare agent plumes over a geographic area of claim 7 wherein saidexpected world model processing algorithm includes a cloud datachange-related examination routine.
 11. The method of detecting andtracking chemical and biological warfare agent plumes over a geographicarea of claim 7 wherein said other sensor analyst processing algorithmincludes a cloud-related data examination routine having data inputsfrom a plurality of other algorithms in said decision engine element.12. The method of detecting and tracking chemical and biological warfareagent plumes over a geographic area of claim 7 wherein said feedbackanalyst processing algorithm includes data inputs from a plurality ofother algorithms in said decision engine element and conditional outputsto a plurality of algorithms in said decision engine element. 13.Chemical and biological warfare agent plumes detection and trackingapparatus comprising the combination of: a plurality of diverse inputcondition responsive warfare agent sensor elements dispersed above ageographic area for detecting and tracking; and a hierarchical array ofintelligent agent decision elements coupled to outputs of said diverseinput condition responsive warfare agent sensor elements; saidhierarchical array of intelligent agent decision elements includingsensor specific first level intelligent agent decision elementsgenerating eXtensible markup language document type definition outputsincluding a cloud identification output and a warfare agent sensorelement output and a confidence output in response to segmented datareceived from said diverse input condition responsive warfare agentsensor elements; said hierarchical array of intelligent agent decisionelements also including a plurality of mid level intelligent agentdecision elements determining cloud-related signal indicia originatingin a warfare agent plume; said hierarchical array of intelligent agentdecision elements also including a plurality of upper level intelligentagent decision elements determining a course of travel for said warfareagent plume in said geographic area.
 14. The method of detecting andtracking a cloud of chemical warfare substance over a geographic area,said method comprising the steps of: coupling an output from each ofdiverse input condition responsive warfare substance sensor elements toa hierarchy of intelligent agent software embodied decision makingalgorithms disposed in an array of lower level, mid level and upperlevel intelligent agents; comparing a plurality of currently receivedchemical warfare substance signatures from said chemical warfaresubstance sensor elements with stored previous similar signatures in anongoing sequence of comparison events accomplished within said hierarchyof intelligent agent software embodied decision making algorithms;upgrading and refining said stored previous similar signatures as saidcomparing step ongoing sequence ensues; communicating data relating tosaid chemical warfare substance signatures; including a data confidencesignal, from intelligent agent decision making algorithms in saidhierarchy of intelligent agent software embodied signal processingalgorithms in a uniform, stable, human comprehensible communicationlanguage; outputting signals identifying presence of said chemicalwarfare substance, movement of a cloud of said chemical warfaresubstance and dispersion of a cloud of said chemical warfare substancefrom said hierarchy of intelligent agent software embodied decisionmaking algorithms in said same uniform, stable, human comprehensiblecommunication language.
 15. The method of detecting and tracking a cloudof chemical warfare substance material over a geographic area of claim14 wherein said hierarchy of intelligent agent software embodieddecision making algorithms include one of Bayesian network, fuzzy logicrules, case based reasoning, probabilistic reasons and genetic algorithmdecision making systems.
 16. The method of detecting and tracking acloud of chemical warfare substance material over a geographic area ofclaim 14 wherein each said intelligent agent in said hierarchy ofintelligent agent software embodied decision making algorithms comprisesa physical representation component, a cognitive component, an agentinterface and a knowledge base.
 17. The method of detecting and trackinga cloud of chemical warfare substance material over a geographic area ofclaim 14 wherein said step of communicating data relating to saidchemical warfare substance signatures, including a data confidencesignal, from intelligent agent decision making algorithms in saidhierarchy of intelligent agent software embodied signal processingalgorithms in a uniform, stable, human comprehensible communicationlanguage includes communicating data according to an eXtensible MarkupLanguage document type definition.
 18. The method of detecting andtracking a cloud of biological warfare substance over a geographic area,said method comprising the steps of: coupling an output from each ofdiverse input condition responsive biological warfare substance sensorelements to a hierarchy of intelligent agent software embodied decisionmaking algorithms disposed in an array of lower level, mid level andupper level intelligent agents; comparing a plurality of currentlyreceived biological warfare substance signatures from said biologicalwarfare substance sensor elements with stored previous similarsignatures in an ongoing sequence of comparison events accomplishedwithin said hierarchy of intelligent agent software embodiment decisionmaking algorithms; upgrading and refining said stored previous similarsignatures as said comparing step ongoing sequence ensues; communicatingdata relating to said biological warfare substance signatures, includinga data confidence signal, from intelligent agent decision makingalgorithms in said hierarchy of intelligent agent software embodiedsignal processing algorithms in a uniform, stable, human comprehensiblecommunication language according to an eXtensible Markup Languagedocument type definition; outputting signals identifying presence ofsaid biological warfare substance, movement of a cloud of saidbiological warfare substance and dispersion of a cloud of saidbiological warfare substance from said hierarchy of intelligent agentsoftware embodied decision making algorithms in said same uniform,stable, human comprehensible communication language.
 19. The method ofdetecting and tracking a cloud of biological warfare substance materialover a geographic area of claim 18 wherein said hierarchy of intelligentagent software embodied decision making algorithms include one ofBayesian network, fuzzy logic rules, case based reasoning, probabilisticreasoning and genetic algorithm decision making systems.
 20. The methodof detecting and tracking a cloud of chemical warfare substance materialover a geographic area of claim 18 wherein each said intelligent agentin said hierarchy of intelligent agent software embodied decision makingalgorithms comprises a physical representation component, a cognitivecomponent, an agent interface and a knowledge base.
 21. The method ofdetecting and tracking a cloud of biological warfare substance materialover a geographic area of claim 18 wherein said step of communicatingdata relating to said biological warfare substance signatures, includinga data confidence signal, from intelligent agent decision makingalgorithms in said hierarchy of intelligent agent software embodiedsignal processing algorithms in a uniform, stable, human comprehensiblecommunication language includes communicating data according to aneXtensible Markup Language document type definition.
 22. The method ofdetecting and tracking chemical and biological warfare agent plumes overa geographic area, said method comprising the steps of: disposing aplurality of input condition responsive warfare agent sensor elementstaken from the group of an infrared radar apparatus, a syntheticaperture radar apparatus and an ultraviolet sensor apparatus above saidgeographic area of detecting and tracking; coupling an output signalfrom each of said input condition responsive warfare agent sensorelements to a sensor-specific first level intelligent agent decisionelement comprising a physical representation component and a cognitivecomponent and an agent interface component and a knowledge basecomponent in a hierarchical array of intelligent agent decisionelements; said intelligent agent decision element physicalrepresentation component comprising an external environment modelinclusive of terrain description data, sun position data, moon positiondata, major water body description data, weather data, soil temperaturedata, soil moisture content data and cloud reflectance data; saidsensor-specific first level intelligent agent decision element furtherincluding a physical representation component, a cognitive component, anagent interface component and a knowledge base component; extracting, insaid sensor specific first level intelligent agent decision element,cloud-related signal indicia from said output, said extracting includingapplying segmented output data and first level intelligent agentknowledge base data to a decision engine algorithm portion of saidsensor-specific first level intelligent agent decision element togenerate a sensor-specific first level intelligent agent decisionelement eXtensible markup language document type definition output; saidfirst level intelligent agent knowledge base data including missionrelated input and output data definitions, intelligent agent decisionelement instructional data and a confidence determination algorithm;said sensor-specific first level intelligent agent decision elementeXtensible markup language document type definition output alsoincluding a cloud identification output and a warfare agent sensorelement output quality-determined first confidence signal; applying saideXtensible markup language document type definition output of saidsensor-specific first level intelligent agent decision element to aplurality of second and successive other mid level intelligent agentdecision elements in said hierarchical array of intelligent agentdecision elements; said knowledge base component and physicalrepresentation component responsive decision element including a currentworld model processing algorithm, an expected world model processingalgorithm, an environment analyst processing algorithm, an other sensorinputs analyst processing algorithm, a feedback analyst processingalgorithm and a memory component each having two way communicationwithin said decision element; said cloud related examination routinghaving environment analyst processing algorithm generated input data,said current world model processing algorithm including a cloud-relatedexamination routing having environmental analyst processing algorithmgenerated input data, said expected world model processing algorithmincluding a cloud data change-related examination routing, said othersensor analyst processing algorithm including a cloud-related dataexamination routing having data inputs from a plurality of otheralgorithms in said decision engine element and said feedback analystprocessing algorithm including data inputs from a plurality of otheralgorithms in said decision engine element and conditional outputs to aplurality of algorithms in said decision engine element; evaluating insaid mid level intelligent agent elements cloud, weather conditions,ground condition and cloud duplicate related outputs received via saidmid level eXtensible markup language document type definition outputs ofsaid first level intelligent agent elements; said evaluating stepincluding a determination of possible natural cause for saidcloud-related outputs, a determination of said cloud-related outputsbeing a warfare agent plume and a determination of a second confidencesignal, said determinations being encoded into an intelligent agentdecision element eXtensible markup language document type definitionoutput; connecting said second intelligent agent decision elementeXtensible markup language document type definition output to aplurality of upper level intelligent agent decision elements in saidhierarchical array of intelligent agent decision elements; anddetermining in said plurality of upper level intelligent agent decisionelements in said hierarchical array of intelligent agent decisionelements an occurrence of a warfare substance attack event and a courseof travel for said warfare agent plume in said geographic area, saiddetermining including determining a third confidence signal level andencoding of said determining step signals into an intelligent agentdecision element eXtensible markup language document type definitionoutput.